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ABSTRACT 



A comparative study was conducted to examine the effects of 
distance delivery on student performance in a science course. Academic 
outcomes and interactions were compared among students (n=44) enrolled in two 
sections of an upper level histology course taught over the course of a 
single semester by the same instructor. Eleven students took the course 
entirely online, while 33 took the course in a traditional, on-campus format. 
Although the performance of both groups on a content pre-test was 
indistinguishable, at the end of the study, students in the online group 
significantly outperformed their peers in the on- campus group on a content 
post -test. The online group also demonstrated a greater frequency of 
interpersonal interactions. Further investigation of the topic and content of 
interactions revealed that although students in the online group initiated a 
smaller percentage of content-related interactions, these interactions 
demonstrated higher levels of thought and abstraction than those generated by 
the on-campus students. (Contains 18 references.) (Author/SAH) 
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An Examination of the Outcomes of a Distance-Delivered Science Course 1 
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ABSTRACT 

A comparative study was conducted to examine the effects of distance delivery on student 
performance in a science course. Academic outcomes and interactions were compared among 
students (n = 44) enrolled in two sections of an upper level Histology course taught over the 
course of a single semester by the same instructor. Eleven students took the course entirely on- 
line, while 33 took the course in a traditional, on-campus format. Although the performance of 
both groups on a content pre-test was indistinguishable, at the end of the study, students in the 
on-line group significantly outperformed their peers in the on-campus group on a content post- 
test. The on-line group also demonstrated a greater frequency of interpersonal interactions. 
Further investigation of the topic and content of interactions revealed that although students in 
the on-line group initiated a smaller percentage of content-related interactions, these interactions 
demonstrated higher levels of thought and abstraction than those generated by the on-campus 
students. 



INTRODUCTION 

A recent report released by the National Center for Education Statistics (1999), indicates 
that the number of available distance-education programs rose 72% during the three-year period 
between 19995 and 1998, with a total of 1,680 institutions offering approximately 54,000 on-line 
courses to 1.6 million enrolled students. A study by the International Data Corporation (1999) 
predicts there will be 2.2 million college students enrolled in distance-learning courses by the 
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year 2002. These rapid growth trends in distance delivered courses underscore the importance of 
investigating the impact of this trend on students. 

When any type of instructional innovation is being evaluated, the first issue that comes to 
mind is student achievement of content objectives. Students must demonstrate at least equal 
achievement (i.e. no significant difference) when learning from the new method as from the 
currently accepted standard, in order for the innovation to be considered for adoption. There are 
a plethora of studies documenting both significant and non-significant differences on student 
achievement (Russell, 1999) when the instructional delivery media is varied. The seemingly 
contradictory results of this body of literature can best by understood in light of a long-running 
debate in the educational technology literature. For over 10 years, Clark (1983, 1994a, 1994b) 
and Kozma (1991, 1994a, 1994b) debated the issue of whether electronic media alone can 
influence learning. While Clark (1983) felt that “media are mere vehicles that deliver instruction 
but do not influence student achievement...” (p. 445), Kozma (1994b) argued “both media and 
methods influence learning and they frequently do it by influencing each other” (p. 1 1). Based 
on this debate and a survey of recent literature (e.g. Joy & Garcia, 2000) it can be concluded that 
media has an indirect effect on learning by facilitating teaching methods which in turn can affect 
learning. Therefore, the current study compared content outcomes for students in both conditions 
simply to verify the viability of the innovation (i.e. on-line delivery does not harm students). The 
impact of the delivery system on student learning was assessed by examining classroom 
interactions for topic and depth of thought. 

The importance of interpersonal interactions in a distance delivered course cannot be 
understated. Faculty-student interaction is one of the cornerstones of effective teaching, 
regardless of delivery medium. For example, student-faculty interaction is recognized as one of 
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the Seven Principles for Good Practice in Undergraduate Education because “[Fjrequent 
student-faculty contact in and out of classes is the most important factor in student motivation 
and involvement. Faculty concern helps students get through the rough times and keep on 
working” (Chickering and Gamson, 1987; p. 4). While this assertion initially targeted traditional 
instruction, it is just as valid, if not more so, for distance delivered courses. Lack of faculty- 
student and student-student interactions lead to feelings of alienation, or “being lost in 
cyberspace” which are commonly cited causes of attrition in distance education (Scott, 1 994, 
Carr, 2000a). In distance education, there are three essential types of interaction, according to 
Moore and Kearsley (1996). These are: learner-content, learner-learner and learner- instructor 
interactions. Learner-content interaction refers to how students interface with the course 
materials. This type of interaction can be affected by the manner in which course materials are 
designed and presented. Learner-instructor interaction encompasses the exchanges that occur 
between the faculty member and students in a variety of modes (i.e. course materials, personal 
communication) and contexts (feedback, encouragement, etc). The functions of this type of 
interaction range from providing opportunities for the students to practice the concepts they have 
acquired, to providing feedback on content mastery and also to mentoring students and 
encouraging them to progress through the course. Learner-learner interactions serve two 
purposes. First, by discussing content with their peers, students are forced to organize and 
postulate their understanding of the concepts under study. This explicit communication exposes 
content misconceptions and allows them to be remedied during the negotiation of meaning that 
takes place while interacting with peers. A second function of learner-learner interactions is to 
allow students to form a community and create a support network. According to Souder (1993), 
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the unique attributes of on-line environments, such as the lack of non-verbal cues, lead students 
to bond with each other more than they would in traditional settings. 

Given the importance of interactions to student success in distance education, this study 
examined interpersonal (i.e. learner-instructor and learner-learner) interactions in order to see 
how their topic and depth of thought was affected by the use of an electronic delivery system. 
Once interactions had been classified by topic, Bloom's classification of questions in the 
cognitive domain (1956) was used to categorize the level of abstraction of content-related 
questions asked in the histology course. 

RESEARCH QUESTIONS 

This study examined the effects of distance delivery on student performance in an upper 
level science (histology) course. 

The specific research questions addressed were: 

1 . Is there a difference in content achievement between students enrolled in face-to-face (on- 
campus) and on-line sections of histology (learner-content interaction)? 

2. How does on-line delivery affect classroom interactions (learner-learner and learner- 
instructor interactions)? 

a) Does the delivery medium impact the number of interactions taking place in each setting? 

b) How does the delivery medium affect the quality of interactions in each setting, as 
measured by Bloom’s Taxonomy? 

3. How does the presence or absence of an instructor affect the number and type of questions 
that occur in on-line group interactions? 
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METHODS 



A hybrid approach was used to examine the effects of distance (on-line) delivery on 
student learning. Observations were conducted in two sections (on-line and on-campus) of the 
same course, taught by the same instructor over the course of one semester. Since the 
participating instructor had sole responsibility for the on-line section, but was part of a teaching 
team for the on-campus course, this study examined student achievement only for the topics she 
presented in both modalities. 

Students enrolled in the on-campus section were observed during regular lecture sessions. 
Students in the on-line section were observed in two settings - on-line chat sessions, where the 
instructor was present, and on-line review sessions, which were student-organized review 
sessions, during which the instructor was not present. 



Figure 1 : Sample content-test questions 



Which of the following statements is false? 

1) The basement membrane separates epithelia from connective tissues. 

2) The basement membrane is composed of Type II collagen. 

3) Parts of the basement membrane are secreted by both epithelia and connective tissue cells. 

4) The basement membrane acts as a selectively permeable membrane. 

Which of the following statements is FALSE? 

1) Adenomeres would be considered part of the parenchyma of a large compound gland. 

2) Merocrine glands are found in only a few locations in the skin of domestic animals. 

3) The cells lining a secretory duct actually contribute to the secretory product of a gland. 

4) Epithelial cells that secrete into the blood stream are referred to as exocrine. 

Which of the following sets of terms are LEAST related? 

1) Spleen: filters blood 

2) Lymph nodes: filter lymph 

3) T-lymphocytes: cell-mediated immune system response 

4) Thymus: primary lymphatic organ 

5) Lymphatic nodules: T-lymphocytes 

Which of the following matches of terms is INCORRECT? 

1) Epidermis: primarily keratinocytes 

2) Dermis: highly innervated 

3) Epidermis: avascular 

4) Hypodermis: dense irregular connective tissue 

5) Dermis: primarily dense irregular connective tissue 
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Students in both sections completed the same pre-test, and the instructor presented the 
same lectures, using the same clinical examples in both situations. Learning outcomes were 
assessed by student achievement on course exams. The same exam questions were asked of 
students in both sections, but at different times during the semester. Student scores on the 
selected questions (Figure 1) were compared by a t-test in order to determine if there were any 
differences in performance between students enrolled in each of the sections. 

The classroom management software used to run the course generated chat transcripts 
and tracked student performance on both pre- and post-tests. An observer manually recorded data 
on on-campus interactions. Transcripts of interactions in each setting were independently coded 
and analyzed by both of the researchers. An interaction was defined as any utterance from a 
participant in the form of a question. Each question was coded into one of four topic categories: 

1. Administrative - questions regarding administrative details of the course, such as due 
dates for assignments 

Examples: “Can you remind us about the 2 ways you would ask a question about secretory 
glands?” “Are there any labs we can attend to review?” “Approx, how many review questions 
will there be on the test?” 

2. Content - any question directly pertaining to course material 

Examples: “Why are erythrocytes anucleate?” “Where are apocrine glands found in animals 
compared to humans?” “What is icterus and what causes it?” 

3. Management - questions used to manage the flow of a class 

Examples: “Hi, Bobby! We just started with a quest to describe a cell that looks like it has the 
potential to divide. Want to give it a try?” “Any of you have hints for Susan?” 
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“Hey, Don, you’ve been quiet. Want to ask a question about lectures 1-4 for the group?” “Does 
anyone want to do a chat the night before the test?” 

4. Social - all questions of a non-academic nature. 

Examples: “Aren’t you glad your names are attached to your answers? No hiding here!” 

“So what’s for dinner? I had left-over spaghetti - not bad.” “Did you have a restful spring break? 
Did you take a break? Read a novel? Lay in the sun? etc?” 

Content questions were further classified according to the demonstrated level of 
abstraction or depth of thought, in accordance with the definitions established in Bloom's 
Taxonomy (Bloom, 1956): 

1. Knowledge: Simple recall or recognition of a concept. E.g. “What are the three layers of 
the basement membrane?” 

2. Comprehension: Interpretation, translation. At this level, the student must be able to 
demonstrate use of abstraction when asked to. E.g. “Brown fat is found in hibernating 
and newborn animals. Why is it brown - anybody?” 

3. Application: Requires application of an abstraction to new problem without being shown 
how to do it in new situation. E.g. “Why are the cells lining blood vessels flat and have 
small, heterochromatic nuclei? 

4. Analysis: Breaking down material into constituent parts and detecting relationship of 
these parts in the whole. E.g. “If you were the Great Creator, now that you’ve 
RECOGNIZED the antigen, what do you want to do with it?” 

5. Synthesis: Putting together parts and elements to elucidate a previously poorly defined 
pattern or structure. E.g. “Testosterone stimulates sebaceous glands, so the cure for ‘stud 
tail’ in cats is ?” 
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6. Evaluation: Judging the extent to which ideas, solutions, methods and materials satisfy 



criteria. E.g. “What might cause a neutrophilia?” 



Data were statistically analyzed via one-way ANOVA and post-hoc comparisons were 
conducted using the LSD test. An alpha value of 0.05 was selected as the significance level for 
all tests. A full description of statistical analysis procedures employed can be found in 
Schoenfeld-Tacher, McConnell & Graham (2001). 

Results: 

While there were no significant differences in academic performance between students in 
either section on the pre-test, students in the on-line section significantly outperformed their 
counterparts in the on-campus section on the post-test. (Table 1). 



Table 1: Academic Performance on Content Tests (percentage scores) for Students in Each 
Course Section 



On-Line On-campus 



Test 


N 


M 


SD 


N 


M 


SD 


df 


t 


pre-test 


6 


10.67 


9.35 


22 


15.09 


11.51 


26 


0.863 


post-test 


11 


80.11 


10.67 


31 


72.78 


10.15 


40 


-2.03* 



* p <0.05 



The average rate of interactions per hour was significantly greater (F = 6.07, p<.01)for 
both on-line environments than it was for on-campus lectures. Students initiated a larger 
percentage of interactions during on-line class sessions than in on-campus sessions (Table 2). 
This difference was statistically significant (F = 6.49, p<.001). However, there was no significant 
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difference in the frequency with which students and instructor initiated interactions within a 
given setting. 



Table 2: Frequency Counts and Total Percentages of Interactions Initiated by Students and 
Instructor in On-Line Classes vs. On-Campus Lectures 





Instructor 




Students 


Section 


Number 


Percentage 


Number 


Percentage 


On-line 


229 


59.0% 


159 


41.0% 


On-campus 


101 


66.4% 


51 


33.6% 



2 

X ~ 2.54, p > 0.05, no significant difference 



The greatest amount of content interactions occurred during on-campus sessions, while 
the largest percentage of social interactions took place during on-line review sessions (Table 3). 
There was no significant difference in the percentage of personal interactions initiated by 
students or instructor in the on-line class sessions. When the combination of initiator and setting 
was examined, it was found that students were most likely to initiate content interactions in a 
lecture setting, followed by a chat, followed by a review. The instructor was more likely to 
initiate a content interaction in a lecture setting than in a chat. In both lecture and chat 
environments, students were more likely to initiate content interactions than the instructor. 



Table 3: Mean Percentage of Interactions per Session Observed in Each Category, by 
Environment and Type of Initiator 



Environment and Person Initiating the Interaction 
Student Instructor 



Topic of Interaction 


On-Line 

Review 


On-Line 

Class 


On-Campus 

Lecture 


On-Line 

Class 


On-Campus 

Lecture 


Management 


28 


20 


0 


32 


18 


Social 


13 


4 


0 


4 


0 


Administrative 


3 


6 


2 


2 


0 


Content 


56 


70 


98 


62 


82 



An examination of the level (Bloom’s Taxonomy) of content interactions showed that 
students in the lecture section generated significantly larger percentages of interactions at both 
the lowest and highest levels than students in the other environments did (Table 4). 



Table 4: Mean Percentage of Content Interactions per Session at Each Level, by Environment 
and Type of Initiator, Using Weighted Data 



Environment Type and Person Initiating the Interaction 



Student Instructor 



Question Level and Type 


On-Line 

Review 


On-Line 

Class 


On-Campus 

Lecture 


On-Line 

Class 


On-Campus 

Lecture 


Low 


Knowledge 


44.2 


36.3 


52.3 


28.3 


27.0 


Comprehension 


33.1 


26.0 


16.3 


34.6 


28.6 


Medium 


Application 


9.6 


16.8 


15.7 


14.4 


25.5 


Analysis 


3.6 


10.6 


13.7 


6.3 


15.6 


High 


Synthesis 


1.2 


10.2 


0 


10.3 


3.3 


Evaluation 


0.6 


0 


2.0 


0.8 


0 



Approximately 75% of interactions initiated by students in review sessions were low- 
level (knowledge and comprehension). The percentages of low and medium (application and 
analysis) questions asked by students in a class setting (on-line or on-campus) were similar to 
each other, but greater than those for review sessions. The only instance in which students 
initiated an appreciable percentage of high level (synthesis and evaluation) questions was the on- 
line chat. The instructor asked a slightly greater percentage of low-level questions in the on-line 
chat sessions than she did in the on-campus lectures. The differences were more striking for 
medium and high level questions. The instructor initiated a much greater percentage of medium- 
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level questions in the on-campus environment and high-level questions in the on-line 
environment. 

CONCLUSIONS 

The teaching methodologies facilitated by the on-line delivery system led to enhanced 
academic outcomes for students in the on-line section relative to those observed for the 
traditional delivery system (lecture), as measured by performance on a content test. These results 
support the assertion that it is possible to effectively teach science at a distance, because on-line 
delivery enhanced learner-content interaction and created improved learning opportunities for 
students relative to on-campus delivery. 

Greater levels of instructor-student interaction were observed in the on-line sessions, 
clearly dispelling any fears of the delivery medium creating a sense of alienation for students. 
Although the difference in type of participant initiating the interactions in each setting is not 
statistically significant, students did generate a greater number of interactions in the on-line 
sessions, and did so without as much prompting as in the lecture sessions, which reflects their 
active engagement with the material under study. There were no social interactions observed in 
the lecture setting, but the instructor was as likely to initiate a personal interaction as the students 
were during an on-line class. These patterns, along with the sense of ‘bonding’ reported by the 
distance students and the instructor illustrate the importance of learner-instructor interactions in 
promoting learning and how these interactions can be used to encourage student participation. 
The instructor influenced the level of questions asked during on-line sessions. Her presence 
shifted the bulk of the questions from the lower levels seen in the review sessions to the more 
balanced distribution seen in the on-line class sessions. 



■13 



BEST COPY AVAILABLE 



Learner-learner interactions are just as important to student success. As expected, 
students generated more social interactions during review sessions than during on-line class 
sessions. These interactions enhanced camaraderie and fostered the creation of a community of 
learners, who provided support for each other during stressful periods. It is interesting to note 
that all of the students who regularly participated in review sessions successfully completed the 
course. Interestingly, a greater percentage of management interactions were observed during the 
review sessions, when the instructor was absent. During these sessions, a student spontaneously 
assumed the instructor’s role and acted as a facilitator/moderator, ensuring that personal 
interactions were kept to a reasonable level, and that review sessions primarily focused on 
content. 

The overall trend in the distribution of questions among lower and higher levels was 
comparable for both the on-campus lectures and on-line classes, although the percentage of high- 
level questions initiated by both students and instructor was greater in the on-line sessions. This 
demonstrates that on-line delivery led to a shift towards high-level questions in a class-type 
environment, perhaps because it allowed more time for students and faculty to think about their 
responses. The use of internet delivery created a more relaxed atmosphere in both of the on-line 
settings, where students were able to ask questions about a variety of topics. 

SIGNIFICANCE OF THE STUDY 

While the literature is replete with studies demonstrating no significant difference when 
two delivery media are compared, (Russell, 1999), this study makes a valuable contribution to 
the body of knowledge about distance education by considering the effects of a delivery system 



on classroom interactions. 



Although the number of students enrolled in distance delivered courses is rapidly 
increasing, the impact of distance delivery, particularly on learning in the sciences, is not yet 
clear. The issue of whether or not it is possible to teach science at a distance without 
compromising academic rigor has been a topic of recent debate in publications such as the 
Chronicle of Higher Education (Carr, 2000b). This study demonstrates that it is possible to teach 
a science course (including laboratories) entirely on-line without any adverse effects on 



academic outcomes. 
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